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ABSTRACT 

This paper documents a successful methodology for the 
validation of data in general and for evaluating educational finance 
data in particular. The report addresses the errors found as a result 
of the independent completion of Part B-1 of the Elementary-Secondary 
General Information System (ELSEGIS) — the Local Education Agency 
Fiscal Report. This survey collects data on beginning balances, 
revenues, expenditures, and ending balances from a sample of 2,500 
school districts across the country. No discussions of the value of 
the instrument or the data reported there are provided, while all 
errors discovered are reported, the major focus is on systematic 
errors, those errors by type or data element that appear with great 
frequency. This study is significant in that it shows the magnitude 
and direction of the errors that may enter a largo-soale survey and 
the steps that could be taken to avoid them. Awareness of the types 
of errors that may be encountered in a survey could permit a 
researcher to design his data collection system so that more accurate 
data could be obtained. (Author/DN) 
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BACKGROUND OF STUDY 



The mainstay o£ research is data, and data conceding educational 
programs are typically collected through surveys. In order to collect 
data on elementary and secondary schools, the U^S. Office of Education 
utilizes the Elementary-Secondary General Information System (ELSEGIS). 
ELSEGIS is a multi-part survey which currently consists of tlie following 
instruments: 

Fart A-1: Local Education Agency Fall Report on Staff and Pupils 
A- 2: State Education Agency Fall Report on Staff and Pipils 

?art B-1: Local Education Agency Fiscal Report 
B-2: State Fiscal Report 

Part C-IA: Fall Report on Schools 

C-IB: Pall Report on Schools (Supplementary Report) 

Part D-1: State Report of Operating and Non-operating Local Public 
School Systems 

D-L: State Fall Estimates of Expenditures and Salary Data for 
Public Sdiools 

The system is conprohensive in tenns of information collected and school 
districts covered. Using a stratified sample of approximately 2,500 
school districts representative of region, metropolitan status code 
(MSG), and enrollment size, variables such as school enix>llment, grade 
level, local education agency (LFA) expenditures, and LEA revenue may 
be estimated for the nation as a whole. 

ihe survey with which this study is concerned is Part B-1, Local 
Education Agency Fiscal Report, This survey collects data on beginning 
balances, revenues, expenditures, and ending balances (as shown in 
Table 1 below) from the sample of 2,500 school districts across the 
country. The reporting format utilized in the analysis of ELSEGIS data 
has typically been to prepare tables and data displays in a descriptive 
manner with a minimal amount of analytical content. The most recent 
analysis plan, however, was based on the identification of the critical 
issues, both current and projected, in education finance. In addition to 
the highly analytical tables, baseline data were presented in a series of 
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descriptive tables to permit continuity in the ELSEGIS reporting effort. 
This infomation, available in reports published by the Office of Education, 
is valuable to educational planners > policy makers , and administrative 
researchers. 

Table 1 

DATA ELEMEOTS - ELSEGIS V FINANCE SURVEY 



Data Elements 



Data Blemouts 



A. Beginning Balances 

1. For Current Ooerations 

2. For Capital Comitments 

3. Total 

B. Revenue from Ix>cal Sources 

1. Taxation and Appropriations 

2. Triition and Transportation Fees 

3. Other 

4. Total 

C. Revenue from IrtetTftediate Sources 
D* Revenue from State Sources 

E. Revenue from Federal Sources 

1. ESEA Title I 

2. ESEA Title II 

3. ESEA Title III 

4. ESEA Title VI 

5. ESEA Title VII 

6. ESEA Title VIII 

7. NDEA Title III 

8. PL-815 

9. PI0874 

10. Head Start 

11. Follow Through 

12 » Vocational Education 

13/ Nntjonal Lunch/Milk Prograjns 

14, ESAP 

IS* Other 

16. Total 
P. Total Revenue Receipts 
G. Total Non-Revenue Receipts 
!!• Incoming Transfers 
I. Total Receipts 

J. Total of All Balances i Receipts > 
and Transfers 



Kl. Adninistration 

K2a. Instructional Salaries/Teachers 

K2b, Instructional Salaries/Other Prof. Staff 

K2c. Instructional Salaries/Non-Prof. Staff 

K2d. Instructional/Total Salaries 

K2e. Other Instructional Expenses 

K2f. Total Instructional Expenses 

K3. Attendance Services 

K4. Health S9rvices 

K5. Pipil Transportation Services 

K6. Operation of Plant 

K7. Maintenance of Plant 

K8. Fixed Charges 

K9. Total Allocation to Pupil Expen. 

KIO. Food Services 

Kll. Student Body Activities 

K12, Community Services 

K13. Total Current Expenditures 

LI* Sites, New Buildings, Add. aiid Imp. 

L2. New Equipment 

L3. Total Capita? Outlay 

KU. Paid on Principal 

Paid for Intorast 
M3a. Paid to School Housing Authority/ 

Principal 

M5b, Paid to School Housing Authority/ 

Interest 
MS, Other 

M6. Total--Debt Service 

N. Outgoing Transfers 

0. Other Expenditures 

P. Returned to Appropriating Authority 

Q» Total Expenditures 

R. Ending Balances 

1. For Current Operations 
.2, For Capital ConmitnKjnts 
3. Total 

S. Total of Expenditures and Ending 
Balances 



• s 
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The basis for the entries made in the ^LSEGIS finance survey is 
Financial Accounting for Local and State School Systems by Paul L. Reason 
and Alpheus L. White, USGPO, Washington, 1957, and referred to as USOB 
Handbook II. This handbook was designed to provide a set of standard 
receipt and expenditure accounts as a foundation for the accurate and 
consistent reporting of school district financial information. While 
designed to be a guide for all states, the use of Handbook II has not 
been mandated and many states have developed and iitplemented their own 
accounting systems. 

As anyone who has dealt with surveys is awar«, not all questions 
are interpreted the same way by different peof>le, nor are all data 
reported accurately. Variations among the accounting systems employed 
by the LEAs and states, differences in the definitions of terms, and 
other reasons caused the quality of some of the ELSEGIS finance data 
to be suspect. Therefore, a study was begun to conduct a field valida- 
tion and an error analysis of the ELSEGIS data to ar:con?)lish three 
objectives: 

1. To examine the existence of both systematic and random errors 
' in the data reported on the ELSEGIS finance form and to 

determine the potential impact on national estimates and 
projections; 

2. To suggest changes in survey procedures, fonns, concepts, 
instructions, etc., in order to reduce the number of errors 
in future surveys; and 

. 3. To identify the needs for further research in validity study 
are&s. 
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METTJODOLOGY 



A thorough understanding of the total data generation and collection 
processes and recordkeeping practices was essential in conducting an 
effective review or data- validation study. Thus, the first step tak^m 
in the performance of this study was the development of a model to serve 
as a guide for project activities. Using operational terms, the model 
was designed: 

It to provide a background for interpreting and evaluating ob- 
served discrepancies in the data validated, 

2. to pix)vide a schematic method for field interviewers to under- 
stand the data generation and collection processes at the LEA 
and SEA levels, and 

3, to translate the initial findings of the project team into a 
preliminary view of tlie data generation and collection 
processes. 

The model that was developed was a response error model • The 
response error model consisted of an identification of the sources and 
types of errors that could enter the data reporting system. The error 
types were identified by reviewing the editing procedures and ent)rs 
discovered in the previously collected and analyzed ELSEGIS data, by 
reviewing the findings of validation studies conducted on similar types 
of surveys, and through a knowledge of the problems* inherent iu the 
state- federal accounting system relationship. The response error model 
was based on a submodel of school district financial operations viewed 
as an information- generating process. 

While the categories of errors are not mutually exclusive, it is 
U5;eM to categorize them for analytical purposes into the following 
types t 

I. Arithmetic Errors 

2» Definitional Errors 

3. Estimation Errors 

4. Formatting Errors 

5. Timing Errors 
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6, Transcription Errois 
?• Lack of Thoroughness 

8. Lack of Source Data 

9. Misunderstood Instructions 

10* Repeat (Carry-Over) Errors- -this includes total line errors 
unless they were arithmetic 

11. Other- -Identified as to cause 

12, Unknown 

1. Arithmetic Errors are errors in the basic addition or subtrac- 
tion employed inl)uilding the ELSEGIS records or where a 
percentage of an LEA figure was incorrectly calculated and 
entered into the ELSEGIS form, 

2. Definitional Inconsisten cies occur when the SEA or LEA utilizes 
a different definition than that used by OB for the same term. 
Examples of this may be seen in the varying definitions for 
"Other Instructional Expenditures," "Nonrevenue Receipts," and 
"Othor Agency Expenditures." 

3. Es timation Errors occur when the respondent had no data at hand 
loi a given data element and was forced to make an estimate, 
Furthei , estimation errors may occur where data are partially 
recorded on SEA or LEA records or where proration methods must 
be enployed, 

4. Formatting Errors are those errors that arise because data are 
maintaihed at the SEA or LEA in a different format from that 
required for completion of the ELSEGIS instrument. These types 
of errors will generally be found in conjunction with one or 
more of the other errors described. 

5. Ti ming Errorg can be said to occur when more accurate data 
Become available after the ELSEGIS instrument is completed. 

6. Tra nscription Errors occur when numbers are incorrectly trans- 
bribed from one fonn to another. 

7. Lack of Thoroughness : This code is used v;hen there appears 
to be no reason for an error other than carelessness on the 
part of the respondent. 

8. Lack of Source Data : Errors of this nature will occur v;hen 
required data are not available to the respondenti 

9. Misunderstood Instructions ! This code is used when the 
respondent did not understand the instructions given for the 
ELSEGIS instrument. 
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10# Repeat (Carry-Over) Errors are said to exist when tKe error 
is solely the result of a previously reported error, and 
generally appears iji a total line, 

!!• Other Error s are all errors for which a cause is kiown other 
than those ^ove, 

12* Ihikria^m Errors are those errors for v^iich the analyst is un- 
able to assign a cause. 

The response error model was inplemented within a methodological 
framework as :>hown on Figure 1. This operational methodology was 
developed to show the logical flow of events in the validation of the 
ELSEGIS data, delineating each step from the start of the fieldwork to 
the publication of final reports. 

The second major step in the validation of the ELSEGIS finance 
data was the development of the analysis plan. This analysis plan was 
designed to direct the search for two distributions of errors: 

• the distribution of errors by questionnaire data element, and 

• the distribution of errors by type of error. 

Consequently, the analysis concentrated on those errors which appeared 
with a high frequency (systematic errors) as opposed to infrequently 
appearing errors (rrindom errors) . In the case of the distribution of 
errors by data element, a review was made of those data elements in 
which errors were most frequently made, regardless of the type of error 
or cause • From this review, it would be possible to determine which 
data element contained the greatest error in the survey and thus deserved 
the most careful review for sii)sequent surveys. A review of the frequency 
of each type of error (arithmetic, trans criptioh, etc.) would allow the 
detemination of the need for special instructions. 

The data collection methodology utilized consisted of m interview 
with the person who originally completed the ELSEGIS instrument for 
1971-72 and an independent canpletion.of the ELSEGIS instrument utilizing 
basic data sources available at the SEA and LEA. In accomplishing this, 
a package of instruments was assembled consisting of interview guidelines, 
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Contact SEAs 
and LEAs 



X 



Begin Field^vork 



Conduct Initial 
Training Session 



Make Initial 
Site Visit 



Print Data Collection 
Materials 



Conduct Second 
Training Session 



Conpare New and 
Old ELSEGIS- -Determine 
Gross and Net 
Difference Rates 



I 



Conduct Site Vis its- - 
Complete ELSEGIS and 
Conduct INterview 



Dv^termine Causes 
of Errors 



Assign Cause to 
Each Error 



Report on Magnitude 
of Errors 



I 



I 



Relate Causes to 
Accounting System 
Factors 



Detemine Potential 
Impact of Errors on 
Policy Decisioni> 



Review Background 
Information on 
SEA/LEA Accounting 
System 



Develop Recommendations 
for Error Elimination: 
Revised ELSEGIS, Special 
State Instructions, etc. 



Figure 1: OPERATIONAL METHODOLOGy. 
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the original ELSEGIS instrument, and a series of worksheets developed to 
allow the analyst to go from the basic data sources at the SEA or LEA 
to the ELSEGIS instrument. 

Tho next step in the study was selection of the sanple of LEAs about 
which data would be collected. The sample selection was acconplished by 
USOE. The sample selection procedures will be discussed more fully 
in the section concerning the full-scale implementation of the study, 
but, briefly, sites were selected such that national projections of the 
data collected could be made. 

Following the selection of the sample, the site visits were conducted, 
the interviews completed, and the basic ELSEGIS data collected. Following 
this , the data were processed and an ELSEGIS instrument was conpleted for 
each of the LEAs in the sample. The final step in the study was to 
conduct the analysis of the data, developing the distribution of errors, 
assigning causes to each, and developing a series of recommendations 
for changes in the ELSEGIS instruments, procedures, and instructions. 
Basic statistical analysis foaised on the computation of gross and net 
difference rates for each item reported in the ELSEGIS survey. Difference 
rates were computed between the validated data and that previously 
submitted by the states. Similar school districts and states have been 
grouped for comparative analyses* 

The study itself was performed in three distinct phases. Design, 
Pilot Test, and Full Scale Implementation. During the Design Phase, 
described above, the approach and materials were developed and pretested 
for their ability to elicit the required information. The Pilot Test 
consisted of a test of the system in 13 school districts in four states. 
The Full Scale Inplemv^ntation was conposed of data collection for 30 
school districts in 20 states. 
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PILOT STUDY 



This section describes the results of the pilot study conducted in 
Iowa, Massachusetts, North Dakota, and Virginia. The purpose of tho 
pilot study was to gain actual field experience with tho study design, 
the materials, and the instructions to interviewers so that appropriate 
changes coidd be incorporated before full implementation. The site 
visits were made by two staff meiTibers to two states each from November 
1972 to April 1973. At the outset of the study, the plan was to 
limit the site visits to the state agencies only, but this was modified 
after the first phase of the pilot/^tudy was conpleted. 

The data reported during ^he^ELSEGIS survey were generally taken 
from statistical records lopdted at the state education agency. Seven 
states forwarded the questionnaires to the local education agencies for 
con?)letion from LEA records, although the same information may have been 
available at the state offices. Of the four states in the pilot study, 
Virginia Mas one that had sent the questionnaires to LEAs. 

The original intent of the project was to evaluate the ELSEGIS III 
survey for the 1969-70 school year. There were several reasons for 
choosing that particular year. First, the school district financial 
data would be collected to approximately match the 1970 Census. In 
connection with this, the ELSEGIS sa/iple was greatly expanded from its 
normal size to 4,800 school districts for that year, and efforts were 
made by the Off ice of Education and the Bureau of the Census to coordinate 
and integrate the data collected by ELSEGIS and the Census. Second, given 
this extra effort, it would be more important than ever to have accurate 
estimates on the national level of differences and errors made in re- 
porting the ELSEGIS data. Finally, the 1969-70 school-year infoimation 
was, at this stage of the study, the latest data published by the Office 
of Education. 

As the study progressed, it was found to be most helpful* if the 
state official could provide the following set of documents: the state 
financial accounting manual for school districts, state guidelines and 
"■-^j^^^ijps ' oi federal aid to education programs available in the state. 



JlilW^^^^ " : ^ " ^ : .■ ■ - ' ^'''^^'^'WIM^ 

vC; \; a state education agency oiganlzation chart, any regional program descrip-^ . ^ 

tlons, copies of school district financial forms for reporting data to ;? 
^he $tate for the districts in the sample, coding instructions- -computer * 

;i| -: (^t otherwise- -for transcribing data from the district forms to the \ 

■ ELSHGIS ff./TTis, and state lists of payments to the districts for federal 1 
programs. , | 

The state accounting manual was IndispensaMe for the con|>arison J 

: j: , of definitions with USOB Handbook 11 on financial accounting which is J 
the basis of the ELSEGIS definitions. In some cases, however, it was ^'3 
found that the definitions were identical, but the states and particu- j 
larly the districts did not follow the definitions. The state federal 
aid simmaries were useful for an overview of the federal programs that 
^ existed in the state and the program names that were used locally. 
For example, Title I of the Elementary and Secondary Edi^ation Act 
would often be called "ESEA Title I," but sometimes merely "Title I" 
and "aid to the economically deprived," and once sinply "P.L, 89-10." x 
Organization charts were used as a guide to determine the other officials % ■ ■ ^ 
that could provide additional information about programs and also to * 
Identify when programs such as vocational education were operated outside * ft 
, ; of the state education departJTtent,. j 
:. Regional program descriptions provided clues as to what other -^'i^;^ --^.-l-^'yviXii 
educational services (and e:iq)ehditures) might exist in the state which 

'>J;H would not be reported on the district financial records. Copies of the 

school district financial reporting forms were obtained to facilitate : V: 

/ ■ the compariscm of data, Thes^ forms plus the coding instructions for 

ELSEGIS provided the means for checking the completed ELSE61S ioms and ^ 
the means for understanding many of the differences which were then found. 
By means of the combination of district reporting forms and the coding 
; i*^t^ruction, errors in the instructions and, in at least one case, errors j 

^ ,.^"1^y%|ffC€^^ progranmer in the implcmeht^ionrof the instructions 




Finally, the stato lists of payments to the districts for federal 
progranis was found to be essential. Many errors were found in reporting 
.federal revenues which pass through state departments of education. 
In this case, the funds can be inaccurately reported or ovetlooked 
altogether when the districts send their annual financial reports 
to the state. The state wouJd frequently complete the ELSEGIS fom 
from the district report without validating the records against the 
original state payments list. ,v ^ 

IXiring the pilot study, the basic attempt of the researcher was v; 
to obtain all possible financial infomuition, and then to delete every- 
thing that was- not essential. The objectives of the study were still - 
being refined at this point. First as accurate information as could be 
obtained for each school district in the sample was desired and we> :>v , - 
wfnted to obtain a full description of the state (artd local) financial 
system from the accounting principles arid manuals utilized to the Uv^il 
of computerized information retrieval currently installed (and planned 
fot) in the state departments. Different people and Offices had \X> h& 
contacted for each of these areas, a point of diminishing rbtums was 
often soon found with additional effort yielding an' 0q\iivalerit gain in 
knowledge about the system. " : -VT. ;=u ; 

^ a result of tho state site visits in the pilbt study, it w^>V 
decided to visit at least some of the school districts in the sample/ .; 
The reasons for this extension of the original design were numerous, but 
a key factor was tho judgment of the researchers that they were obtaining 
neither a full description of accounting practices which wero used by 
thd districts to report their financial information to the states nor 
the ntost accurate data which might bo available. The pilot study was 
tho proper place to evaluate these judgjnents and, consequehtly, nine of 
the 13 school districts in the pilot were visited. 

The results of the school district site visits were much as had 
beenVp#dicted. Additional errors were found beyond 'those induced 
or transmitted at the stato l^vel, but no systematic patterns were 
■ IM^I^^ als^totin^ guide! ines wore of t¥l^#dif ied to meet 



local needs or practices. And finally it again was found tha^ the point 
of diminishing returns was quickly reached. IVhile many figures could be 
checked against the local books in five or ten minutes, other data would 
require a nunijer of days to verify, reviewing unorganized ledgers and 
account books on a page-by-page basis. At this point, it was determined 
that the outer limits of the scope of the study had been obtained. 

A nuTber of insoluble problems were found. For example, some 
items wore reported on the basis of conputer printouts which at the time 
of the evaluation study had either been destroyed or could no longer be 
found--just the opposite of the usual case where more complete and accur- 
ate data can be obtained by reviewing data after it has been audited. 
Some accounts were either lost or had not been kept in the first place. 
In other cases, individuals would disagree on the correct identification 
of a particular account. Occasionally, two or more sets of data would 
exist, with little guidance supplied by state or local officials i\s 
to which data were best or more accurate. This was the case where a 
school district had a highly automated accounting system on an accrual 
basis, but the state required the annual reports to be submitted on a 
cash basis. In that particular situation, the conversion of the district's 
accrual report to a cash basis report introduced a number of errors that . 
were ultimately reported on tlie BLSEGIS survey. 

Two major aspects found in the pilot study which may have an inpact 
on financial surveys in the future were school district •'dependen^'!^ ,■:;;> 
and the existence of regional services . ttependency consisted of shared 
services between the school district and the city or county in which it 
was located. Many items fron health service? to operation and maintenr 
anco of the school plant are currently being supplied by non- school 
political jurisdictions, and, in the opinion of the researchers > there 
are sharply increasing instances of these shired services. Regional 
^01' i^lUtH^ services w6 re also observed in all states in the 
"liridt computer centers, media centeri5| and tralnS4??pfcialists 

•^p/ovide services' td many districts which are not entefi^' iii dirimt 



in th6 school district. This, too, seems to be a growing trend, and 
there appear? to be no easy way to reflect these services in surveys 
of school district finances. 

The pilot study established the content and limits of what could 
be obtained in field visits at the state and school district level. 
Interview procedures were streamlined as the various protocols became 
established and as our knowledge increased as to the items most likely 
to be found at the state and district levels. The categories of errors 
and differences of opinion reflected in the BLSEGIS reports were 
described in the pilot study report and few additional categories were 
found in the full-scale implementation. From the experience of the 
pilot study, it was decided that a thorough visit to all the school 
districts in the sample would consume an inordinate amount of time and 
resources to be a useful part of the full-scale stu^. Instead, it was 
decided that an attenpt would bo made to visit only one district in each 
state in the san^lo, in order to verify the findings made at the state 
level and to spot-check the accuracy of various accounts, but to spend 
only one day in the school district. Finally it was determined that the 
second major purpose of the study, to develop detailed descriptions of 
state financial reporting systems, could be fully obtained by the state 
level visits coupled with a visit to a single school di$trict. 

Finally, it should be noted that there was a change from the evalua- 
tion of ELSEGIS in (1969-70 school year) to ELSEGIS V (1971-72 school 
year) as a result of the pilot study. Particularly in the detail and 
scope of the field work, this kind of rigorous examination of the ELSEGIS 
financial survey, often considered to be the best survey of school district 
finances in the nation, has never been done before. As a result of the 
chance to examine and evaluate comparable financial records and reporting 
systems at both the state and school district level, more time was spent 



FULL SCALE IMPLEME>frATION 



Building on the pretest described above, the fliU scale impleioenta- 
tlon of the validation study occurred in a ^0 state, 30 school district 
sample* The data collection took place through site visits. One or two 
professionals visited each site selected, conducting the one-hour interview 
and couplet iJig the ELSEGIS form from the state's basic data sources. The 
amount of time required for this effort varied from two to ten man-days 
per state depending on size and conplexity. The major thrust of the site 
visits was to gather data which could bo coii?>ared to data previously 
submitted by the state as well as to learn directly the problems which 
the state encountered in the completion, of the ELSEGIS finance instrument. 

In addition to the visit to the state education agency, a site 
visit was made to one of the sample school districts in each state in 
order to determine the availability of the required financial data at 
that level. The function of this visit was to determine whether the 
school districts would be better able to complete the ELSEGIS fortns 
than the state education agv^ncies, as well as to substantiate SEA 
results, ' 

The sanple of school districts about which c(ata were collected in 
the full scale implememation of the study was selected by tho Office Y k :{ 
of Education, The universe included all school districts with an en* 
rollment of 300 or greater that previously completed the ELSEGIS in- ^ 
strument for 1971-72 with the exception of Nevf York City (because of 
the unwarranted time requirements due to coit?)lexity) and lotvav^ 
Massachusetts, North Dakota, and Virginia (which wete included in the 
pilot test). The original ELSEGIS samplo was subsampled down to make 
the cells self-weighting, and differential weights were applied accord- 
ing to school district size and degree of urb^ization. Table t shows 
fthO s^le selection procedure \vnd T^ble 3 shows the School dis'tf ietSf 



Table 2 

ELSEGIS Sample Selection Procedure 



Pall 1971 
PntoUmont 



25,000* 



10,000-24,99^) 



S,000- 9,999 



2>500- 4,999 



500- 2,499 



MSC 1 



(a) 



0))/c(c) 



lOO^^'VS 



5 • 



124/2 



MSC 2 



MSC S 



70/3 



272/2 



595/S 



$89/3 



10 



1824/2 



123/1 



427/2 



1006/2 



11 



S195/S 



MSC 1 ^ Motropolitan Status CoM I * Central. City 
KBC 2 • H?trppoUtari Status Co^ 2 » O^ef Urfean 
MSC i - Metropolitan Status Codo 5 r Rural 



MM 



&mpl0 CoU . 
NuiTtor irt universe 
Nunbor selected 



The first step in the validation of the data submitted on the EI^E01$ 
finance survey was to conduct an interview with the individual \^ho ha^X^^ 
previously conpleted the form, Hie interview was structured thrlugh'^^the ;^ 
use of an interview guideline which consisted of 13 questions fo'feusing 
on some of the known areas of confusion as well as on errors occurring 
most frequently on previous surveys. In addition, data were collected 
concerning background information on the staters accounting system 
(such as the degree of conputerization and the use df U^B Handbook II), 
This intdfview procedure was designed to d'etedt and d65Ci*ibe*tte? 
prih^iii41;'causes of error in the original survey. For exait5>le| a check , 
was' fi^de^tfe determine the extent to which federal and stat6 deHrii£ioris 
W^1^9;^^Mi|BRt, ' Furth^f i'%dn exact esipertdiliji^o' Infonfiatldn was not 
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Table 3 



ELSEGIS Validation Study Sample 

State, School District 

Alabama, Montgomery 

Arkansas. Port Smith 

California, China Lake Joint Elementary 

California, Palm Springs Unified 

Connecticut, Hamden 

Florida, Volusia County 

Georgia, Greene County 

Illinois, Rockford 

Illinois, Southeastern Comm. Unit 

Kentucky, Ldtcher 

Louisiana, Livingston Parish 

Michigan, Ann Arbor 

Michigan, Gerrish Higgins 

Michigan, Lansing 

Minnesota, Robbinsdale 

Missouri, Berkeley 

Missouri, Carthage R-9. 

New Mexico, Los Alamos 

New York, Amherst 

New York, Lake Placid 

New York, Sewanhaka 

North Carolina, Mecklenburg County 

Ohio, Bexley 

Ohio, Perry 

Oklahoma, Davenport 

Oregon, Hillsboix) 007. 

Texas, Ector Co. ISD 

Texas, Junction Co, ISD 

Texas, Pasadena ISD 

Washington, Highline 



Pall 1971 
Enrollment 

37,425 
12,994 
2,443 
6,690 
9,930 
32 i 519 
2,522 
41,010 
628 
5,637 
10,187 
18,538 
1,100 
30,825 
27,983 
4,458 
3,569 
5,134 
3,028 
1,080 
U 682 
80,047 
2,608 
lj841:. 

370 
2,595 
24,025 

858 1: 
35,276 : 
27,910 



available from a state's records an attempt was made to deteni&ne 
whether the method of estimation or proration was soundly based and 
whether it was unifotmly applied. 

The second step in the validation study was to conpleto an BISEGIS 
form for each LEA in the sample from the state's basic data sources. In 
going from the basic data sources to a properly completed ELSEGIS form, 
a series of worksheets and their accompanying instructions were developed. 
The rationale for this was twofold: first, to ensure the collection of 
accurate data across LEAs and, second, to ensure consistency among the 
various analysts that were used In the data collection effort, It 
was the purpose of this package to specify the steps necessary in the 
transference of existing source data from original documents at the SEA 
or LEA to the ELSEGI3. The basic school district reporting documents -ivp: 
were reviewed and, where state and federal definitions did not coincide, 
the state handbooks were examined in depth. While the federal accounting 
i. handbook (Handbook II) has been in existence for 17 years, not all states 
have adopted the system of accounts contained in it. . In these cases, 
states generally had developed a translation device Either to allow' them 
to move between th,e two charts of accounts <>t between the state's chart 
1^ of accounts and tl^e ELSEGIS instrument.; Tho^^ trmSlation d^V 

reviewed carefully as they frequently were a source of error. Once tho 
basic source data were obtained from state files, an ELSEGIS Instrument 
s- was completed for each of the state's sample school districts. The data 
thus developed were compared against that previously submitted by the 
state and any discrepancies were studied until resolved, 

llie data processing which was performed consisted of two phases. 
The first phase was to complete a case study fOr each of the sites 
visited. This case study was designed to provide qualitative infor- 
mation about the SEA/LEA, the federal programs in existence, and the 
M^tlonship of the ELSEGIS Instrument to the local accounting methods, 
\?|^^tline was developed which allowed thO analyst to go fit>m the data 

t^^Bliiii^' ■^iM:M4^^H|--resr^^ 
ilil^B^^^illPg- aspeof ^'f^i^ 



STUDY o/mrrs 



J . A nunber of significant outputs have resulted from this study, 
A package of site visit reports documenting the ELSEGIS completion 
process in each state visited has been prepared along with an overall 
sunmary. Among the findings that resulted from a review of the site 
visit reports are: 

1. Joint state- federal program^ are often reported incorrectly- - 
For programs that were completely funded by state or federal sources, the 
reporting of data is manageable by the states and school districtSi 

v:- Those programs that are funded jointly by the state and federal govern- 

• J ment present a serjous problem for respondents, however, for the 6i^>port 

u of programs such as school lunch and vocational education, school districts 
■' f typically receive one check fix>m the state with neither the source of 
■ ' funds nor the portion provided by the state and federal governments 

shown. The result of this is that school districts typically report the 
programs as funded entirely out of state or federal sources, with the 
tendency toward the latter. While state records will generally show 
the distinction, these data are only available by soli<:itng data' from a 
separate office, a practice that is frequently not done when the original 
li;.: ELSEGIS report was prepared, 

K^'' 2, Some programs are not reported- -This observation was true in a. 

; r . . nuiTber of the states visited. The individual, who conpleted the ELSEGIS 

• • form would enter all the programs that could be discovered (usually the 
[i:. large programs in terms of dollars), but this often meant that some 

" programs, especially the small ones, were missed. Progrsims that were not 
reported include Drug Education, National Forest-Shared Revenues, ESEA 
Title n, and Vocational Education. 

3. Programs Operated by intermediate • agencies are ' f requeritly not 
; • t^^ted--lh)less%e school district- is directly responsible for ttie 

HSS^^P^iijsfe^jiiifth^i^^^ 

^B^M^^wiiiiiiisitiiiiiM 



the district had no Infonmation and for which the state had only aggre- 
gate data, This was a problem for validation of the ELSEGIS as the 
states and LEAs had few records to Indicate the existence of these 
programs and only through considerable research were they discovered. 
There is no way to be certain that all of the programs were reporto4 
even with the effort that was put forth, 

As previously stated, the approach which RMC followed in the data 
analysis was to search for two distributions of errors; 

• the distribution of errors by' questionnaire data element, and 

• the distribution of errors by type of error. 

Consequently, analysis was concentrated on those errors that appeared 
with a high frequency, In the first case, data elements in which errors 
were most frequently maide regardless of the type of error or cause were 
reviewed. Frcm this? review, it was possible to determine which data 
elements contained the greatest error in the ELSEGIS finance survey and 
thus deserved the most careful review in ^ture sui'Veys. A review of the 
frequency of appearance of each type of error (arithmeUc, transcription, 
etc.) allowed the determination of whether a need existed for special 
instructions. Difference rates were confuted betwe0n the validated 
data and the state -reported data, Mienevef there were differences be- 
tween the actual data and the reported data, a resolution interview was 
held to detemine the cause of the discrepancy. Examples of Ihe types of 
inconsistencies between the state reports and federal reporting require- 
ments include: 

Estimation Error- -Midwestern state^ 

The staff salaries reported by this state are estimated, not actual 
expenditures. They are reported on their annual reporting form in this 
manner. 



1. Because of a guarantee of confidentiality of data reported and 
because each state had discrepancies that could have been iistisd here, 
Itap names have not been identified. 



Fomatting Errot--Great Lakes state 

The form from which this state draws the data to prepare the 
ELSEGIS reports does not contain lino items for PL-815 and National 
School Lunch among other federal programs. Data for these programs are 
available from either a separate report or directly from the school 
districts themselves. • 

Timing- -Southern s tate 

The ELSEGIS reports for this state are completed from the school 
district financial report. The form is required te be submitted by the 
LEAs to the state by July 15, but the last of the reports usually 
arrived in October, These forms are then audited, a procedure which was 
not completed for the fiscal year ended June 30, 1972, until August 2, 
1973. As the ELSEGIS form was to be submitted to the Office of Education 
by March 15, 1973, audited data were not reported. 

Lack of Thoroughncss--Northwcstem state 

While this state's reporting form is highly detailed, errors in the 
coding instructions designed to transfer data from the form to the 
ELSEGIS instrunent resulted in receipts from PL- 874 and NDEA Title III 
being reported under "Other Federal," Federal Forest Funds being reported 
under PL- 874, and no entries made on the NDEA Title III line. 

Lack of Source Data- -Western state 

For the repayment of state building fund debt by the local school 
districts, a single payment is made with no designation of the portion 
of the funds that is either for interest or principal. The state, in 
reporting on the ELSEGIS form, arbitrarily applied the division of 
601 principal, 40% interest to the total amount. 

Misunderstood Instructions --Northwestern state 

As this state is moving toward the adoption of a program budgeting 
approach to accounting for school finance, retirement and social security 
payments made on behalf of pupil transportation workers were reported 
under Pupil Transportation rather than fixed charges. 

In addition to the above types of outputs, five products remain to 
be developed as the study is concluded. 

1. All data coi%>arisons will be completed and gross and net ef'ror 
rates for each data item will bo confuted. 

, |. x;.::; Estimates rof;!^ naticittalj^^llll^ll^l^l^ and 



3, Recommendations fbr revised accounting and data reporting 
method? will bo developed, Those reconmenditions will be designed to 
affect the relationship between the states and the federal government 
vii^-a-vis the ELSEGIS survey alone and will not presume to revise 
accounting methodology on a broader scale. 

4, The relationship of the frequency or magnitude of the errors 
to other state or school system characteristics vrill be investigated. 
Among the variables to be investigated will be school district size» 
degree of computerization of accounting facilities, and the use of 
federal accounting definitions. Standard multiplo regression tehcniques 
will be applied. 

5, A thorough description of the financial accounting systems in 
the 24 states visited will be prepared. This could serve as the basis 
for the preparation of individual ELSEGIS instructions for each state 
as well as a guideline for the connparison of educational revenues and 
expenditures on a state-by-state basis. 



The primary purpose of this paper was to document a successful 
'methodology for the validation of data in general and for evaluating • . 
^^diphtational finance data in particular. Secondly, the paper was designed' 
to caution against the unquestioning acceptance of financial data as 
reported, even data sibjected to c&reful audit procedures. The report 
addresses the errors found as a result of the independent conpietion 
of the ELSEGIS instrument for a sample of. school districts. As such, 
the report dealt only with what is wrong with the ELSEGIS instrument 
and did not discuss the value of- the instrument and the data reported 
therefrom, \ihile reporting all errors discovered, the major effort 
was concentrated on systematic .errors, those errors by type or data 
element which appear with great frequency. 

The significance of this. study is th^t it shows the magnitude and 
direction of the errors that' may enter a large-scale survey and the steps 
which may be taken to avoid them. Awateness of the types of errors 
that may be encountered in a survey will permit tjie researcher to 
design his data collection system so that more accurate data may be 
obtained. 



